Condense testing and pre-commit workflow into single workflow#4545
Conversation
|
I think I'm actively working against you here 😆 Let's see if you can catch me up.
|
|
We know we saturate our Github Action runners at every opportunity, we need to be more frugal with spinning up a new machine. Each new workflow requires a whole new runner, which needs to hit the APIs more times, which uses quota, money etc. By putting it in the same workflow, it can share variables, secrets, input triggers, etc. which means fewer things to maintain and go wrong. Generally, people create a new workflow when they want to create a new job. This is a single step workflow, which does a related thing to the test workflow (lint). My rule of thumb is if it uses the same trigger they should be in the same workflow. We could even put this in the same job as the linting to make this more efficient. It's only 608 lines because it duplicates all code across pytest and nf-test and it has 1 million exceptions for conda in both. Once you remove that it's only ~230 lines which is pretty low for a monorepo like this. |
nf-core/modules#4545 (comment) Co-authored-by: adamrtalbot <adamrtalbot@users.noreply.github.com>
edmundmiller
left a comment
There was a problem hiding this comment.
Awesome, thanks for the detailed explaination, I'm caught up now!
Documented it here nf-core/ops@a76929c for other to stumple on.
|
Let's take it to it's natural conclusion: #4554 |
nf-core/modules#4545 (comment) Co-authored-by: adamrtalbot <adamrtalbot@users.noreply.github.com>
My war on workflow bloat continues.